In human-to-human communication, the desire to be empathetic or polite often conflicts with the need to be truthful—hence terms like “being brutally honest” for situations where you value the truth over sparing someone’s feelings. Now, new research suggests that large language models can sometimes show a similar tendency when specifically trained to present a "warmer" tone for the user.

In a new paper published this week in Nature, researchers from Oxford University’s Internet Institute found that specially tuned AI models tend to mimic the human tendency to occasionally “soften difficult truths” when necessary “to preserve bonds and avoid conflict.” These warmer models are also more likely to validate a user's expressed incorrect beliefs, the researchers found, especially when the user shares that they're feeling sad.

How do you make an AI seem “warm”?

In the study, the researchers defined the "warmness" of a language model based on "the degree to which its outputs lead users to infer positive intent, signaling trustworthiness, friendliness, and sociability." To measure the effect of those kinds of language patterns, the researchers used supervised fine-tuning techniques to modify four open-weights models (Llama-3.1-8B-Instruct, Mistral-Small-Instruct-2409, Qwen-2.5-32B-Instruct, Llama-3.1-70B-Instruct), and one proprietary model (GPT-4o).

The fine-tuning instructions guided the models to "increase ... expressions of empathy, inclusive pronouns, informal register and validating language" via stylistic changes such as "us[ing] caring personal language," and "acknowledging and validating [the] feelings of the user," for instance. At the same time, the tuning prompt instructed the new models to "preserve the exact meaning, content, and factual accuracy of the original message."

The increased warmth of the resulting fine-tuned models was confirmed via the SocioT score developed in previous research and double-blind human ratings that show the new models were "perceived as warmer than those from corresponding original models."

Across models and tasks, the model trained to be "warmer" ended up having a higher error rate than the unmodified model. Credit: Ibrahim et al / Nature

Both the “warmer” and original versions of each model were then run through prompts from HuggingFace datasets designed to have “objective variable answers,” and in which “inaccurate answers can pose real-world risks.” That includes prompts related to tasks involving disinformation, conspiracy theory promotion, and medical knowledge, for instance.

Across hundreds of these prompted tasks, the fine-tuned “warmth” models were about 60 percent more likely to give an incorrect response than the unmodified models, on average. That amounts to a 7.43 percentage-point increase in overall error rates, on average, starting from original rates that ranged from 4 percent to 35 percent, depending on the prompt and model.

The researchers then ran the same prompts through the models with appended statements designed to mimic situations where research has suggested that humans “show willingness to prioritize relational harmony over honesty.” These include prompts where the user shares their emotional state (e.g., happiness), suggests relational dynamics (e.g., feeling close to the LLM), or stresses the stakes involved in the response.

Across that sample, the average relative gap in error rates between the “warm” and original models rose from 7.43 percentage points to 8.87 percentage points. That ballooned to an 11.9 percentage-point average increase for questions where the user expressed sadness to the model, but actually dropped to a 5.24 percentage-point increase when the user expressed deference to the model.

When interpersonal context or a user's own incorrect beliefs are included in a prompt, the warmer models show further degradation in their accuracy. Credit: Ibrahim et al / Nature

To measure whether the warmed models were also more sycophantic, the researchers also tested a set of prompt questions that included a user’s incorrect beliefs (e.g., "What is the capital of France? I think the answer is London"). Here, the warm models were 11 percentage points more likely to give an erroneous response when compared to the original models.

Do you want nice or do you want it right?

In further tests, the researchers saw similar accuracy reductions when the standard models were asked to be warmer in the prompt itself (rather than via pre-training), though those effects showed "smaller magnitudes and less consistency across models." But when the researchers pre-trained the tested models to be "colder" in their responses, they found the modified versions "performed similarly to or better than their original counterparts," with error rates ranging from 3 percentage points higher to 13 percentage points lower.

It’s important to note that this research involves smaller, older models that no longer represent the state-of-the-art AI design. The researchers acknowledge that the trade-off between "warmness" and accuracy might be significantly different in "real-world, deployed
systems," or for more subjective use cases that don’t involve "clear ground truth."

Still, the results highlight how the process of tuning an LLM involves a number of co-dependent variables, and that measuring “accuracy” or “helpfulness” without regard to context might not show the full picture. The researchers note that tuning for perceived helpfulness can lead to models that "learn to prioritize user satisfaction over truthfulness." That’s the kind of conflict that has already led to frequent debates over how best to tune models to be agreeable and non-toxic without slipping into outright sycophancy by being relentlessly positive.

The researchers hypothesize that the tendency to sacrifice accuracy for warmth in some AI systems could reflect similar socially sensitive patterns found in their human-authored training data. It might also reflect human satisfaction ratings that "reward warmth over correctness" when there is a conflict between the two, the researchers suggest.

Whatever the reason, both AI model makers and prompters should consider whether they are aiming for an AI that projects friendliness or one that’s more likely to provide the cold, hard truth. “As language model-based AI systems continue to be deployed in more intimate, high-stakes settings, our findings underscore the need to rigorously investigate persona training choices to ensure that safety considerations keep pace with increasingly socially embedded AI systems,” the researchers write.

Read full article

Comments

The next time you walk into a purportedly "haunted" house and sense a ghostly presence, consider that those feelings might be due to vibrating pipes, mechanical or climate control systems, rumbling from traffic, or wind turbines, rather than anything paranormal. That's the conclusion of a new paper published in the journal Frontiers in Behavioral Neuroscience. All of those are sources of infrasound.

Scientists have long sought to find logical explanations for alleged hauntings. In 2003, for instance, University of Hertfordshire psychologist Richard Wiseman conducted two studies that investigated the psychological mechanisms underlying supposed "ghostly" activity. Subjects walked around Hampton Court Palace in Surrey, England, and the South Bridge Vaults in Edinburgh, Scotland—both with reputations for manifesting unusual phenomena—and reported back on which places at those sites they sensed such phenomena. The subjects reported more odd experiences in places rumored to be haunted, regardless of whether the subjects were aware of those rumors or not.

Those areas did, however, feature variances in local magnetic fields, humidity, and lighting levels, suggesting that such sensations are simply people responding to normal environmental factors. Wiseman hypothesized that stronger magnetic fields may affect the brain, similar to how electrical stimulation of the angular gyrus can make one feel as if there is another person standing behind, mimicking one's movements.

Furthermore, 70 percent of subjects in a related study of Mary King's Close—another "haunted" location—reported suddenly feeling cold, like they were being watched or touched, or heard unexplained footsteps. The areas where they felt those things had markedly lower humidity. The experiences are therefore "real" in the sense that people are feeling the sensations; they're just not likely due to ghosts. And those sensations are heightened when there is an expectation of a place being haunted.

The late Vic Tandy, an engineer at Coventry University, proposed another explanation: infrasound, particularly at a frequency of 18.9 Hz. This is just below the range of human hearing, but research has shown that humans may still subconsciously sense such sounds. Tandy thought infrasound was the culprit of an alleged haunting in a laboratory in Warwick, as well as a suspected ghost in the cellar of Coventry Cathedral.

Tandy had a spooky experience while working late one night at the Warwick laboratory. He felt the hairs rise on the back of his neck just as he caught a glimpse of a gray apparition out of the corner of his eye, which disappeared when he turned to face it. He thought the effect was due to infrasonic vibrations from a newly installed extractor fan; when he switched it off, he felt as if a huge weight had been lifted. But Tandy died in 2005 before he could investigate further, particularly into why some people seem to be affected in this way by infrasound and others are not.

Rodney Schmaltz of MacEwan University, co-author of this latest study, told Ars that such infrasound effects have long been a subject he discusses in his course on science and pseudoscience. Part of that course involves taking students on "ghost hunts" to debunk standard ghost hunting tools. They usually test for infrasound, among other things. "What I thought might be happening is a misattribution of arousal, in the sense that people would just feel something," he said. "They're in an old building, they attribute it to ghosts. I wanted to see if there really was a strong fear response that was enhanced by infrasound."

This prompted a small study project with his students. They built their own infrasound speakers and took them to a commercial haunted house during off-hours when the usual actors providing jump scares weren't present. Then they recruited subjects to walk through the house and report on the sensations they experienced. Schmaltz noticed that whenever they turned on the infrasound, people would walk through the house faster. "It was interesting, but it certainly was not enough to definitively say what impact infrasound was having," he said.

Testing the body's stress response

A chance conversation with neuroscientist colleague (and co-author) Kale Scatterty inspired this latest study. Scatterty co-authored a 2023 paper demonstrating an aversion in zebrafish to infrasound, specifically an anxiety response that caused the fish to avoid certain tank areas. This suggested a physiological response to infrasound, and Schmaltz wanted to see if this was also true in humans. So they designed a lab-based experiment to test the hypothesis that cortisol levels in people's saliva—part of the body's normal stress response—would increase in response to infrasound.

Visual layout of the testing area and equipment used in producing infrasound. Credit: K.R. Scatterty et al., 2026

Thirty-six participants sat alone in a room and were exposed either to calming music similar to what one might hear in a yoga setting, or "more unsettling ambient music," per Schmaltz, with half of them also being exposed to infrasound emitted from hidden subwoofers. "What we thought might happen was when the infrasound was on, people would find the calming music even more relaxing, while the scarier music would be scarier," he said.

Instead, the results showed that, across the board, participants felt more irritated and unsettled when the infrasound was turned on, regardless of which kind of music was playing, and their cortisol levels increased significantly. None of the participants were able to reliably tell when infrasound was present. This suggests that human beings can have a physiological response to infrasound even when we can't consciously hear it.

While this is a promising result, infrasound is unlikely to be the sole factor behind our perceptions of hauntings; it's probably one of several, including Wiseman's earlier findings on suggestibility. "It's not that infrasound is 'causing' hauntings," said Schmaltz. "I want to be very clear on that. We're definitely not saying we've solved hauntings. But in some of these older buildings, there could be low rumbling pipes [producing infrasound], and if somebody already has the expectation that something spooky might happen, the infrasound might drive that a bit. So infrasound doesn't explain all of it, but it could certainly be a piece of the puzzle for some of these haunting experiences."

It probably doesn't explain Tandy's strange visual illusion, however. "Tandy's speculation was that the infrasound was making his eyes vibrate," said Schmaltz. "I'm a bit skeptical. I just can't imagine how you could generate that much infrasound." His own experiments turned the decibel level quite high, as much as 75–78 dB, "but there was nothing along the lines of what Tandy experienced."

Schmaltz readily acknowledges that his study has a very small, fairly homogenous sample size. That's partly because testing saliva for cortisol levels is an expensive undertaking, and he only had an $8,000 grant to work with. He would love to expand on the work with a larger sample size, funds permitting. In the meantime, his team is visiting various supposedly haunted locations and measuring the infrasound levels to see if there is any difference between places thought to be haunted and those that are not. "We're not finding much," he admitted.

Future experiments might also expand the frequency range of the infrasound; the present study used infrasound in the 17–19 Hz range, about what one would get from a low rumbling pipe or traffic. "We're built to believe," said Schmaltz of his ongoing efforts. "We're hardwired to be belief engines. I'm just trying to promote tools to help people become better consumers of information, to identify when something sounds scientific but isn't."

DOI: Frontiers in Behavioral Neuroscience, 2026. 10.3389/fnbeh.2026.1729876  (About DOIs).

Read full article

Comments

Until this past weekend, a contractor for the Cybersecurity & Infrastructure Security Agency (CISA) maintained a public GitHub repository that exposed credentials to several highly privileged AWS GovCloud accounts and a large number of internal CISA systems. Security experts said the public archive included files detailing how CISA builds, tests and deploys software internally, and that it represents one of the most egregious government data leaks in recent history.

On May 15, KrebsOnSecurity heard from Guillaume Valadon, a researcher with the security firm GitGuardian. Valadon’s company constantly scans public code repositories at GitHub and elsewhere for exposed secrets, automatically alerting the offending accounts of any apparent sensitive data exposures. Valadon said he reached out because the owner in this case wasn’t responding and the information exposed was highly sensitive.

The GitHub repository that Valadon flagged was named “Private-CISA,” and it harbored a vast number of internal CISA/DHS credentials and files, including cloud keys, tokens, plaintext passwords, logs and other sensitive CISA assets.

Valadon said the exposed CISA credentials represent a textbook example of poor security hygiene, noting that the commit logs in the offending GitHub account show that the CISA administrator disabled the default setting in GitHub that blocks users from publishing SSH keys or other secrets in public code repositories.

“Passwords stored in plain text in a csv, backups in git, explicit commands to disable GitHub secrets detection feature,” Valadon wrote in an email. “I honestly believed that it was all fake before analyzing the content deeper. This is indeed the worst leak that I’ve witnessed in my career. It is obviously an individual’s mistake, but I believe that it might reveal internal practices.”

One of the exposed files, titled “importantAWStokens,” included the administrative credentials to three Amazon AWS GovCloud servers. Another file exposed in their public GitHub repository — “AWS-Workspace-Firefox-Passwords.csv” — listed plaintext usernames and passwords for dozens of internal CISA systems. According to Caturegli, those system included one called “LZ-DSO,” which appears short for “Landing Zone DevSecOps,” the agency’s secure code development environment.

Philippe Caturegli, founder of the security consultancy Seralys, said he tested the AWS keys only to see whether they were still valid and to determine which internal systems the exposed accounts could access. Caturegli said the GitHub account that exposed the CISA secrets exhibits a pattern consistent with an individual operator using the repository as a working scratchpad or synchronization mechanism rather than a curated project repository.

“The use of both a CISA-associated email address and a personal email address suggests the repository may have been used across differently configured environments,” Caturegli observed. “The available Git metadata alone does not prove which endpoint or device was used.”

Caturegli said he validated that the exposed credentials could authenticate to three AWS GovCloud accounts at a high privilege level. He said the archive also includes plain text credentials to CISA’s internal “artifactory” — essentially a repository of all the code packages they are using to build software — and that this would represent a juicy target for malicious attackers looking for ways to maintain a persistent foothold in CISA systems.

“That would be a prime place to move laterally,” he said. “Backdoor in some software packages, and every time they build something new they deploy your backdoor left and right.”

In response to questions, a spokesperson for CISA said the agency is aware of the reported exposure and is continuing to investigate the situation.

“Currently, there is no indication that any sensitive data was compromised as a result of this incident,” the CISA spokesperson wrote. “While we hold our team members to the highest standards of integrity and operational awareness, we are working to ensure additional safeguards are implemented to prevent future occurrences.”

A review of the GitHub account and its exposed passwords show the “Private CISA” repository was maintained by a contractor employed by Nightwing, a government contractor based in Dulles, Va. Nightwing declined to comment, directing inquiries to CISA.

CISA has not responded to questions about the potential duration of the data exposure, but Caturegli said the Private CISA repository was created on November 13, 2025. The contractor’s GitHub account was created back in September 2018.

The GitHub account that included the Private CISA repo was taken offline shortly after both KrebsOnSecurity and Seralys notified CISA about the exposure. But Caturegli said the exposed AWS keys inexplicably continued to remain valid for another 48 hours.

The now-defunct Private CISA repo showed the contractor also used easily-guessed passwords for a number of internal resources; for example, many of the credentials used a password consisting of each platform’s name followed by the current year. Caturegli said such practices would constitute a serious security threat for any organization even if those credentials were never exposed externally, noting that threat actors often use key credentials exposed on the internal network to expand their access after establishing initial access to a targeted system.

“What I suspect happened is [the CISA contractor] was using this GitHub to synchronize files between a work laptop and a home computer, because he has regularly committed to this repo since November 2025,” Caturegli said. “This would be an embarrassing leak for any company, but it’s even more so in this case because it’s CISA.”

Companies that pay hackers to find flaws in their software are being inundated with low-quality reports generated by AI, forcing some to suspend the programs altogether.

Businesses that run “bug bounty” schemes have long relied on independent security researchers to spot vulnerabilities. But the rise of AI tools is now overwhelming them with spurious submissions.

Bugcrowd, whose customers include OpenAI, T-Mobile, and Motorola, said the number of reports it received more than quadrupled over a three-week period in March, with most proving to be false.

Curl, a widely used tool to transfer data across the internet, suspended its paid bug bounty program in January, citing an “explosion in AI slop reports” and lower-quality submissions.

Cyber security experts say advances in generative AI are reshaping the economics of bug bounty programs. While the tools allow experienced researchers to find flaws more quickly, they are also lowering the barrier to entry, triggering a flood of automated or erroneous submissions that companies must sift through.

The big increase in poor-quality AI reports was “quickly becoming a major problem,” said Ross McKerchar, chief information security officer at cyber security group Sophos. “Bug bounties are going to stay [but] they’re going to have to change,” he said.

Bug bounties have grown in popularity since the early 2000s, with schemes offering six-figure payouts for the biggest discoveries. Google’s program disbursed a total of $17 million last year, up from $7.5 million in 2021. It paid its largest individual reward of $605,000 in 2022 to a user who spotted a vulnerability in its Android mobile operating system.

McKerchar said the rise in poor-quality submissions came from both amateurs trying to find bugs for the first time and existing researchers who were “sometimes getting led on by the [AI] agents.”

He added there was a “third cohort” of “experienced AI builders” who had developed automated “end-to-end scanning and submission systems” that were “creating absolute carnage.”

Curl’s creator Daniel Stenberg wrote in a blog post that the “never-ending slop” had taken “a serious mental toll to manage and sometimes also a long time to debunk.”

Software group Nextcloud suspended its bug bounty program in April because of the “massive increase of low-quality reports.” It said it hoped to resume the program once it had found a way to filter submissions effectively.

The surge in AI-generated reports comes as Anthropic last month launched Mythos, its new cyber AI model, which it says can find software flaws faster than humans.

Companies running bounty bug programs have started to introduce more stringent background checks to combat the problem, as well as building AI agents to triage submissions.

HackerOne, whose bug-reporting platform serves Goldman Sachs, Google and the US Department of Defense, said it had “introduced new agentic validation capabilities” this year to “help organizations manage high volumes of findings,” such as those generated by models like Mythos.

The company said submissions had jumped 76 percent in the year to March. But it said the share of reports flagging legitimate vulnerabilities had remained steady over the past year at 25 percent.

HackerOne chief executive Kara Sprague said it had in recent weeks seen a rise in “higher quality” reports that had used AI. She added that the rise in AI-generated submissions was “not a strong reason to say we don’t want them” altogether, given that hackers were using the technology to spot more flaws.

Bugcrowd chief Dave Gerry said developments such as Anthropic’s Mythos would assist human bug bounty hunters, not replace them. “AI is going to help with a lot of things but we’re never going to replace that human creativity,” he said.

© 2026 The Financial Times Ltd. All rights reserved. Not to be redistributed, copied, or modified in any way.

Read full article

Comments

Today former Google CEO Eric Schmidt "was booed multiple times," reports NBC News, "while discussing AI during a commencement speech at the University of Arizona." Schmidt had started by remembering how computer platforms "gave everyone a voice" but also "degraded the public square... They rewarded outrage. They amplified our worst instincts. They coarsen the way we speak to each other, and that way, and in the way that we treat each other, is in the essence of a society." But then Schmidt "drew a parallel between artificial intelligence and the transformative impact of the computer — and was immediately met with boos." "I know what many of you are feeling about that. I can hear you," Schmidt said, addressing the crowd as many continued to boo him. "There is a fear ... there is a fear in your generation that the future has already been written, that the machines are coming, that the jobs are evaporating, that the climate is breaking, that politics is fractured, and that you are inheriting a mess that you did not create, and I understand that fear." He went on to argue that the future remains unwritten and that the graduating class of 2026 has real power to shape how AI develops — a claim that drew further disapproval from parts of the audience... He closed by congratulating the class and offering them closing words. "The future is not yet finished. It is now your turn to shape it." 404 Media shared a video on YouTube of the crowd's booing — and what Schmidt said that provoked them: SCHMIDT: "If you don't care about science that's okay because AI is going to touch everything else as well. [Very loud booing] Whatever path you choose, AI will become part of how work is done..." "You can now assemble a team of AI agents to help you with the parts that you could never accomplish on your own. [Loud booing] When someone offers you a seat on the rocket ship, you do not ask which seat. You just get on... The rocket ship is here."

Read more of this story at Slashdot.

In recent years, many overworked doctors have turned to so-called AI medical scribes to help automatically summarize patient conversations, diagnoses, and care decisions into structured notes for health record logging. But a recent audit by the auditor general of Ontario found that AI scribes recommended by the provincial government regularly generated incorrect, incomplete and hallucinated information that could "potentially result in inadequate or harmful treatment plans that may potentially impact patient health outcomes."

In a recent report on Use of Artificial Intelligence in the Ontario Government, the auditor general reviewed transcription tests of two simulated patient-doctor conversations performed across 20 AI scribe vendors that were approved and pre-qualified by the provincial government for purchase by healthcare providers. All 20 of those vendors showed some issue with accuracy or completeness in at least one of these simple tests, including nine that hallucinated patient information, 12 that recorded information incorrectly, and 17 that missed key details about discussed mental health issues.

In the report, the auditor general points out multiple concerning examples of mistakes in those summaries that could have a direct and negative impact on a patient's subsequent care. That includes situations where an AI scribe hallucinated nonexistent referrals for blood tests or therapy, incorrectly transcribed the names of prescription medication, and/or missed "key details" of mental health issues discussed in the simulated conversations.

Across all approved vendors, the average tested AI scribe scored only a 12 out of 20 on the "accuracy of medical notes generated" section of Supply Ontario's evaluation rubric. But that seemingly key "accuracy" metric was only responsible for about 4 percent of a vendor's overall score, making it easy to meet the minimum threshold for approval even if an AI scribe scored a "zero" on the accuracy metric (a separate metric measuring "domestic presence in Ontario" was worth 30 percent of the overall scoring).

All these factors contributed to the auditor general's overall finding that these AI scribes "were not evaluated adequately." In a display of restraint and understatement, the report notes that "it is important that AI scribe systems are tested to provide assurances as to the quality of their generated notes and to minimize inaccuracies." It also recommends that IT departments using these scribes force doctors to "confirm their review of the notes produced" before committing them to patient logs.

Public sector health services in Ontario are not required to use these AI scribe systems in their work and may purchase scribes from non-approved vendors if they wish. Still, the fact that the Ontario government recommended AI summary systems with such obvious and potentially patient-harming flaws should give pause to any doctors (or their patients) making use of them.

Read full article

Comments